feat: add ResponseFormat support to call_model_input_filter (#3563)#3569
feat: add ResponseFormat support to call_model_input_filter (#3563)#3569ylzn567 wants to merge 4 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c9ef7dc95d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 29b8bb6f51
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| context_wrapper=context_wrapper, | ||
| input_items=input, | ||
| system_instructions=system_prompt, | ||
| response_format=None, |
There was a problem hiding this comment.
When a call_model_input_filter tries to inspect or replace ModelInputData.response_format, this call always seeds it with None, and the returned filtered.response_format is never used when model.get_response(..., output_schema=output_schema) is invoked later. In that scenario the new API is effectively inert: filters cannot see the current structured-output format and any strict JSON/schema format they return is dropped before the model request; the streamed path has the same response_format=None call site.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Note
To use Codex here, create a Codex account and connect to github.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f6f503e86
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| previous_response_id=previous_response_id, | ||
| conversation_id=conversation_id, | ||
| prompt=prompt_config, | ||
| response_format=filtered.response_format, |
There was a problem hiding this comment.
When any run reaches this call, model is an agents.models.interface.Model, but the interface and built-in implementations still define get_response(..., prompt=...) without a response_format keyword, so Python raises TypeError: unexpected keyword argument 'response_format' before the model request is made. The streamed path has the same new keyword at line 1489, so this needs the interface and every implementation updated (or the keyword removed until supported).
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Note
To use Codex here, create a Codex account and connect to github.
…nding maintainer guidance
|
Hi team! 👋
To cleanly resolve this, the Since updating the entire model interface layer expands the scope of this PR significantly, I've temporarily reverted the kwarg injection. I would love your architectural guidance: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 17b8225a26
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| context_wrapper=context_wrapper, | ||
| input_items=input, | ||
| system_instructions=system_prompt, | ||
| response_format=current_response_format, |
There was a problem hiding this comment.
When a call_model_input_filter returns a modified ModelInputData.response_format, the value is still discarded: this new argument only seeds the filter, while the subsequent model call continues to use the original output_schema and never reads filtered.response_format (the streamed path does the same). Fresh evidence in this revision is that the earlier crash was removed, but the returned response format remains unused, so filters cannot actually enforce or disable a JSON schema despite the new API field.
Useful? React with 👍 / 👎.
|
I see @jordanchendev has already provided a complete solution along with the requested contract tests over in #3571! Closing this PR to keep the tracker tidy. It was a great learning experience diving into the run loop and input filter architecture. Keep up the great work! |
1 participant
Closes #3563
Description
This PR adds
response_formatsupport to thecall_model_input_filtercallback inRunConfig. This allows developers to dynamically modify the expected response format (e.g., enforcing a strict JSON schema) immediately before the model is called.Changes Made
response_format: ResponseFormat | Noneto theModelInputDatadataclass insrc/agents/run_config.py.maybe_filter_model_inputfunction insrc/agents/run_internal/turn_preparation.pyto accept and pass theresponse_format.src/agents/run_internal/run_loop.pyto correctly injectmodel_settings.response_formatinto the filter during both streamed and non-streamed runs.Next Steps
(If you have a specific existing test fixture or file you recommend I use as a reference for testing this serialization/resume behavior, I'd appreciate the pointer!)